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ABSTRACT 


This  paper  describes  an  airborne  deployment  of  the  Viper  sniper  detection  and  counterfire 
system1  and  several  improvements  that  were  incorporated  after  a  review  of  comments 
from  users  of  the  fielded  systems2.  The  Viper  sensor  was  flown  aboard  a  T-34  aircraft 
and  the  Naval  Research  Laboratory  (NRL)  Sparrow  unmanned  aerial  vehicle  (UAV). 
Infrared  (IR)  and  visible  video  sequences  were  recorded  for  post  processing.  Algorithms 
for  motion  compensation,  frame  registration  and  target  detection  were  then  developed  to 
run  on  the  collected  image  data.  No  inertial  attitude  data  were  recorded  and  all  motion 
compensation  functions  were  made  using  image  data.  These  experiments  indicate  that 
automatic  detection  of  muzzle  flash  from  an  airborne  platform  is  possible,  and  techniques 
that  were  developed  for  background  estimation  and  false  alarm  reduction  with  a 
stationary  sensor  can  apply  with  modifications  to  a  moving  sensor. 


1  Caulfield,  J.T.;  Gower,  P.W.;  Moroz,  S.A.,  Burchick,  D.A.,  Ertem,  M.C.,  Pierson,  R.B.;  “Performance  of  the 

Vectored  Infrared  Personnel  Engagement  and  Return  Fire  (VIPER)  IRFPA  Muzzle  Flash  Detection  System” (U), 
Proceedings  of  the  IRIS  Specialty  Group  in  Passive  Sensors,  1996 

2  Gower  P.W.,  Moroz  S.  A.,  Burchick  D.A.,  Ertem  M.C.,  Pierson  R.B.  “The  Vectored  Infrared  Personnel  Engagement 

and  Retumfire  (VIPER)  System  and  Its  Counter  Sniper  Application  ”(U),  Proceedings  35th  IRIS  Symposium  on 
Infrared  Countermeasures,  1998 
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An  improvement  that  was  made  on  the  original  Viper  system  was  the  integration  of  an 
acoustic  detector  to  use  muzzle  blasts  to  verify  IR  detections,  to  obtain  target  range,  and 
to  reduce  false  alarms. 


Another  improvement  was  to  change  the  camera-rifle  calibration  technique  which 
originally  required  the  identification  of  three  known  points  in  an  infrared  image.  Two 
new  methods  were  developed:  one  with  a  camera  mounted  scope,  and  a  second  using  a 
visible  light  camera  boresighted  with  the  IR  sensor  to  allow  selecting  calibration  points  in 
the  visible  spectrum. 


1.0  INTRODUCTION 

The  Viper  system  detects  the  muzzle  flash  of  a  sniper's  weapon  with  a  focal  plane  array  staring  midwave 
infrared  camera.  The  video  signal  from  an  Inframetrics  MilCam  is  fed  to  a  Datacube  MV-250  real-time 
image  processor.  Detection  algorithms  incorporate  a  histogram  based  automatic  gain  and  pedestal  control 
routine,  a  temporally  matched  flash  filter,  spatial  filters,  adaptive  background  estimation,  adaptive 
thresholding,  track  filtering  and  a  final  detection  locator.  The  Viper  system  that  was  described  in  earlier 
publications  (1,2)  was  designed  for  use  with  a  stationary  camera.  Thus,  if  the  camera  was  moved  or  jarred 
the  system  would  produce  a  very  large  number  of  false  alarms  while  the  camera  motion  lasted. 

It  was  decided  to  test  the  feasibility  of  using  the  Viper  system  from  moving  platforms,  such  as  aircraft  and 
land  vehicles.  Algorithm  modifications  for  airborne  deployment  are  generally  easier  to  implement  than 
for  a  land  vehicle.  There  are  two  main  reasons  for  this.  The  first  is  that  motion  characteristics  of  an 
airborne  platform  can  be  considered  to  be  smoother  than  that  of  a  land  vehicle.  Thus,  the  camera  is 
subjected  to  less  of  a  jarring  motion  if  placed  on  an  aircraft.  The  second  reason  is  that  if  deployed  on  an 
aircraft,  the  objects  in  the  field  of  view  are  essentially  in  the  far  field.  Nearby  objects,  such  as  trees  or 
buildings  within  the  camera  field  of  view  complicate  deployment  on  moving  land  vehicles. 

Objects  in  the  near  field  of  view  make  algorithm  design  for  a  moving  camera  system  significantly  more 
difficult.  This  is  because  as  the  camera  is  translated,  the  relative  motion  of  objects  in  the  far  field  and  in 
the  near  field  are  not  identical,  resulting  in  new  areas  of  the  scene  being  occluded  or  becoming  visible 
before  the  camera.  For  example,  if  the  camera  is  deployed  on  a  Hummv  driving  down  a  road,  trees  in  the 
near  field  will  result  in  areas  in  the  background  which  will  alternately  become  visible  and  then  be  blocked 
from  the  camera.  Since  the  Viper  system  uses  an  adaptive  background  estimation  method  for  signal 
detection,  these  effects  of  near  field  objects  would  have  to  be  remedied.  One  approach  would  be  to 
implement  a  routine  to  estimate  a  range  to  each  object  in  the  image,  essentially  producing  a  three 
dimensional  map  of  the  scene.  Algorithms  of  this  type  to  derive  three  dimensional  information  from  a 
moving  camera  have  been  developed  by  others3.  However,  these  methods  require  more  processing  power 
than  available  in  the  current  Viper  system.  Therefore,  it  was  decided  to  begin  implementation  of  a  moving 
camera  Viper  system  with  an  airborne  version. 


3  A.  Murat  Tekalp,  "Digital  Video  Processing"  (U),  Prentice  Hall,  1995 


2.0  AIRBORNE  DEPLOYMENT 


The  first  task  in  modifying  the  Viper  system  for  airborne  operation  was  to  modify  the  detection 
algorithms  to  compensate  for  the  image  changes  due  to  camera  motion.  Since  all  objects  seen  by  the 
camera  are  essentially  at  the  far  field,  successive  video  fields  could  be  registered  to  each  other  just  by 
using  the  warping,  translation  and  rotation  operations.  Furthermore,  by  using  a  relatively  narrow  field  of 
view  lens  (32  degrees  x  24  degrees)  pincushion  distortion  was  minimized  and  the  need  for  the  image  warp 
operation  was  eliminated.  For  the  initial  flight  tests  translation  and  rotation  operations  were  sufficient  to 
register  successive  image  frames. 

The  Viper  temporal  filter  is  a  four  frame  long  finite  impulse  response  matched  filter.  This  means  that  as 
each  frame  of  video  is  acquired  (at  60  frames  per  second)  it  has  to  be  registered  with  the  three  previous 
frames,  each  of  which  is  offset  by  its  own  translation  and  rotation  parameters.  Furthermore,  the 
background  activity  estimate  has  to  be  updated  at  each  frame  by  the  translation  and  rotation  parameters 
for  that  frame. 

There  are  several  ways  to  estimate  the  motion  of  the  video  camera  from  one  frame  to  the  next.  One 
method  is  to  use  an  attitude  measurement  device,  such  as  an  inertial  measurement  unit  (IMU)  or  fiber 
optic  gyro,  to  measure  the  motion  of  the  camera;  and  applying  the  known  mechanical  relationship 
between  the  camera  and  the  attitude  reference  to  calculate  the  translation  and  rotation  parameters.  This 
would  probably  be  the  approach  taken  in  the  design  of  an  operational  system,  where  it  is  likely  that  an 
attitude  reference  is  already  built  into  the  aircraft. 

A  second  approach  is  to  use  the  information  contained  in  the  video  signal  itself  to  estimate  camera  motion 
parameters.  This  is  done  via  correlation  methods  to  compare  two  successive  frames  and  find  translation 
and  rotation  parameters  that  give  a  best  match.  Since  in  Viper  the  target  to  be  detected  is  a  point  source, 
the  presence  or  absence  of  the  flash  would  not  significantly  effect  the  correlation  based  image  registration 
routines.  The  problem  however  is  that  these  methods  require  significant  amounts  of  processing  power  and 
were  considered  not  feasible  with  the  existing  hardware. 

While  the  problem  of  frame  registration  was  being  considered  another  issue  became  apparent.  The 
payload  size  and  weight  limitations  of  the  NRL  Sparrow  UAV  allowed  only  a  sensor  package  and  a 
downlink  to  be  flown.  The  integration  of  an  IMU  package  with  its  own  downlink  was  not  practicable.  It 
was  thus  determined  that  video  only  could  be  downlinked  and  recorded.  Camera  motion  estimation  would 
be  performed  only  based  on  downlinked  video. 

It  was  desired  to  collect  the  downlinked  video  in  digital  format.  To  do  this  a  personal  computer  (PC) 
based  video  recorder  was  designed  and  built.  This  system  used  a  frame  grabber  and  a  dual  channel  wide 
SCSI  architecture  to  strip  incoming  video  to  four  streams,  each  of  which  was  recorded  on  a  separate  hard 
disk,  in  a  redundant  array  of  inexpensive  drives  (RAID)  implementation.  This  custom  developed  RAID 
recorder  solution  was  chosen  as  a  practical  solution  since  it  was  much  less  expensive  than  purchasing  a 
commercial  off  the  shelf  RAID  recorder.  This  recorder  also  provided  a  live  display  with  overlaid  frame 
number  and  timestamp  which  was  recorded  on  to  video  tape  along  with  an  audio  track.  Radio 
communications  and  input  from  a  microphone  were  recorded  on  this  audio  track,  greatly  simplifying 
flight  test  documentation  and  providing  ground  truth  data  as  to  when  each  shot  occurred. 

Test  flights  were  flown  at  a  small  private  airfield  at  Airlie,  Virginia.  On  the  days  of  testing  the  airport  was 
closed  to  all  other  traffic.  The  system  was  first  tested  on  a  T-34  aircraft.  An  advantage  of  this  aircraft  is 
that  it  is  a  tandem  seater  with  a  canopy  that  can  be  opened  in  flight.  The  camera  was  handheld  with  the 
operator  holding  it  steady  out  the  port  side  of  the  aircraft.  The  operator  also  was  responsible  for  setting 


the  pedestal  and  gain  of  the  camera  by  monitoring  the  IR  video  through  a  viewfinder.  To  make  integration 
with  the  T-34  as  simple  as  possible  two  battery  belts  were  used  to  power  the  camera  and  UHF  video 
transmitters.  The  transmitters  and  antennas  were  attached  to  the  inside  of  the  canopy  using  suction  cups. 
This  allowed  an  unobstructed  line  of  sight  to  the  ground  receiver  antenna. 

To  make  the  motion  estimation  problem  easier  it  was  decided  to  place  marker  beacons  on  the  ground  near 
the  area  of  interest.  These  simply  consisted  of  four  coal  fueled  grills  that  were  on  the  ground  and  which 
were  lit  before  each  flight. [Figure  1]  These  provided  a  fixed  frame  of  reference  for  objects  on  the  ground. 
For  each  frame  of  video  the  location  of  each  marker  beacon  could  be  automatically  tracked  based  on  its 
position  on  the  previous  frame.  A  very  simple  correlation  tracker  searching  a  small  area  was  implemented 
to  automatically  track  each  beacon  on  the  image. 

The  ground  station  consisted  of  a  vehicle  mounted  video  recorder  system  and  the  UHF  receiver  system 
with  an  18  inch  dish  antenna.  One  person  was  assigned  the  task  of  manually  pointing  the  receive  dish  at 
the  aircraft  as  it  flew  overhead. 

During  each  flight  the  pilot  would  keep  the  aircraft  at  a  constant  altitude  and  roll  angle,  and  by  varying 
power  would  control  his  position  to  orbit  the  target  area  in  a  circular  path.  The  ground  team  consisted  of  a 
recorder  operator,  an  antenna  tracker,  a  rifleman,  and  a  safety  coordinator. 

When  the  recorded  data  was  analyzed  and  algorithms  for  motion  compensation  based  on  the  four  marker 
beacons  were  written  it  was  apparent  that  the  automatic  detection  of  rifle  muzzle  flashes  from  an  aircraft 
is  feasible.  It  was  decided  to  continue  on  to  demonstrate  the  deployment  of  Viper  on  a  UAV. 

For  data  collection  flights  from  the  UAV  platform  required  knowing  whether  the  shooter  was  within  the 
field  of  view  of  the  sensor  and  to  time  rifle  shots  appropriately.  In  flight  tests  with  a  manned  aircraft  the 
camera  operator  essentially  kept  the  camera  pointed  at  the  target,  and  was  in  radio  communications  with 
the  ground.  With  a  UAV  however,  ground  personnel  would  not  easily  have  known  if  the  sniper  area  was 
within  the  sensor  field  of  view.  Additionally  it  was  desired  to  provide  a  means  of  visible  feedback  to  the 
UAV  operator  to  guide  him  in  establishing  the  flight  path.  To  do  this  a  visible  light  video  camera  was 
incorporated  into  the  sensor  package  with  its  own  downlink  [Figure  2].  The  two  video  streams  (IR  and 
visible)  were  synchronized  to  each  other  by  using  the  RAID  recorder  frame  counter  and  mixing  this  with 
the  visible  video  which  was  recorded  on  tape.  The  visible  tape  would  later  be  digitized  with  the  RAID 
recorder  and  a  frame  accurate  (at  30  frames/sec)  alignment  with  the  corresponding  infrared  video  would 
be  made  [Figure  3].  Furthermore,  the  two  cameras  were  boresighted  and  camera  coordinates  were 
calibrated  for  further  analysis. 

The  data  collected  at  the  UAV  test  flights  were  analyzed  by  running  the  automatic  marker  beacon 
tracking  and  detection  algorithms.  The  detections  from  the  infrared  sequences  were  then  overlaid  as  target 
symbology  onto  the  visible  video  collected  during  the  flights.  This  was  done  by  using  the  cross  camera 
calibration  parameters.  This  allowed  the  tracking  of  the  shooter  location  once  it  was  detected  as  long  as  it 
and  the  marker  beacons  were  within  the  field  of  view. 

These  experiments  demonstrated  that  the  Viper  system  can  be  effective  when  deployed  on  an  airborne 
platform  and  that  algorithms  can  be  developed  to  detect  and  track  the  location  of  muzzle  flashes  from 
manned  and  unmanned  aircraft.  [Figure  3]  Although  in  this  case  the  algorithms  were  developed  to  run 
offline  with  recorded  data,  their  real  time  implementation  is  practical. 


3.0  IMPROVEMENTS  TO  VIPER  SYSTEM 


In  addition  to  airborne  deployment  several  improvements  to  the  original  Viper  system  were  implemented. 
In  1996  four  Viper  systems  were  delivered  for  evaluation  by  the  US  Army  Dismounted  Battlespace 
Battlelabs  and  by  the  US  Marine  Corps.  Upon  receiving  comments  from  the  counter  sniper  evaluation 
teams  several  design  enhancements  were  made.  One  of  these  was  the  incorporation  of  an  acoustic  detector 
to  verify  IR  detections  and  to  provide  passive  range  information  for  a  muzzle  flash.  Another  improvement 
was  the  development  of  two  new  simplified  methods  for  calibrating  counterfire  rifle  coordinates  to  the 
camera. 


3.1  Acoustic  Ranging  and  Verification 

A  simple  acoustic  detector  was  built  with  a  COTS  microphone,  an  amplifier  and  a  PC  mounted  data 
acquisition  system.  The  microphone  output  was  sampled  at  6000  Hz  and  a  digital  filter  was  implemented 
to  estimate  the  energy  of  the  acquired  signal.  A  simple  infinite  impulse  response  filter  was  used  to 
estimate  background  energy  levels  and  to  detect  the  presence  of  a  muzzle  blast.  Since  this  acoustic  filter  is 
to  be  used  in  conjunction  with  Viper,  and  to  verify  IR  based  detections,  it  was  considered  important  not  to 
miss  any  detections.  Since  the  acoustic  detector  would  not  be  the  primary  sensor  its  stand  alone  false 
alarm  rate  was  considered  to  be  of  secondary  importance.  Thus  while  the  acoustic  filter  may  give  false 
alarms  from  car  doors  closing  and  other  impulsive  noises,  it  was  considered  that  in  an  operational 
situation  with  a  counter  sniper  team  these  would  not  degrade  the  performance  of  the  primary  IR  flash 
detection  sensor. 

The  Viper  system  alarm  output  was  connected  to  the  PC  and  each  IR  detection  opened  a  predetermined 
range  gate.  Knowing  the  speed  of  sound  at  the  ambient  temperature,  if  an  acoustic  detection  was  received 
within  the  range  gate  the  detection  was  considered  to  be  verified,  and  the  range  calculated  to  the  target  by 
measuring  the  flash-bang  delay  time. 

The  system  was  tested  at  the  Fort  Meade,  Maryland  rifle  range  and  demonstrated  that  acoustic  verification 
for  IR  detection  is  highly  feasible.  One  of  the  unforeseen  situations  encountered  was  that  on  one  of  the 
test  days  an  adjacent  rifle  range  was  in  use  with  rapid  fire  weapons  and  that  the  acoustic  signatures  from 
the  other  range  were  being  detected.  Thus,  in  environments  where  other  weapons'  blasts  are  audible 
within  the  range  gate  triggered  by  an  IR  event  the  usefulness  of  acoustic  verification  may  be  diminished. 

Work  is  currently  underway  to  implement  the  acoustic  detection  algorithms  on  a  digital  signal  processor 
platform  for  incorporation  into  the  Viper  system. 

3.2  Camera  to  Rifle  Calibration 

The  Viper  system  has  a  magnetometer  /  inclinometer  which  attaches  to  counterfire  rifle.  The  calibration 
procedure  was  to  choose  three  points  in  the  infrared  video  display.  The  observer  moved  a  cursor  onto 
these  points  in  the  infrared  image  and  had  the  shooter  point  his  rifle  at  the  chosen  points.  A  comment 
encountered  during  the  evaluation  of  the  system  was  that  under  some  situations  it  was  difficult  to  find 
three  points  which  showed  up  at  high  contrast  on  the  IR  image.  It  was  therefore  decided  to  develop 
alternative  procedures  to  select  three  calibration  points. 

The  first  approach  that  was  implemented  was  to  attach  a  second  video  camera  in  the  visible  spectrum  to 
the  IR  camera.  This  would  allow  the  observer  to  choose  aim  points  in  the  visible  light  image.  The  two 
cameras  were  cross  calibrated  (by  the  same  procedure  as  used  in  the  UAV  deployment)  and  the  rifle  to 
visible  calibration  would  provide  the  rifle  to  IR  camera  calibration. 


This  system  was  built  and  evaluated  in  the  field.  While  it  worked  well  under  most  lighting  situations, 
there  were  cases,  such  as  when  positioned  toward  the  sun  or  looking  at  very  high  contrast  scenes,  when  it 
was  deemed  to  be  unacceptable. 

To  solve  the  calibration  problem  another  approach  was  developed.  This  involved  the  construction  of  a 
miniature  gimbal  with  azimuth  and  elevation  axes  that  incorporated  a  standard  sniper's  scope.  This 
removable  assembly  was  attached  to  the  camera  and  was  operated  in  the  same  manner  as  a  theodolite.  By 
aiming  the  calibration  scope  at  three  fixed  points,  the  camera  coordinates  of  these  points  were  determined. 
The  shooter  aimed  at  these  points,  giving  the  magnetometer  /  inclinometer  readings  for  these  calibration 
points.  Thus  the  system  was  calibrated. 

The  scope  based  calibration  consistently  produced  accurate  results.  And  since  the  calibration  device  is  a 
sniper's  scope,  identical  to  the  one  on  the  shooter's  rifle,  it  always  provided  an  unambiguous  calibration. 
The  only  negative  side  to  this  approach  is  that  there  is  one  extra  piece  of  equipment,  the  gimballed 
calibration  head,  as  part  of  the  system.  The  calibration  gimbal  which  was  manufactured  as  a  prototype  is 
rather  unwieldy,  but  this  item  can  be  miniaturized  for  production  versions. 


4.0  CONCLUSIONS 

The  feasibility  of  infrared  detection  and  location  of  muzzle  flashes  from  an  airborne  platform  has  been 
demonstrated  and  significant  improvements  have  been  made  to  the  Viper  system  to  enhance  its 
performance  on  the  ground.  A  study  of  the  application  of  these  methods  to  land  based  vehicle  deployment 
is  currently  under  way. 
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Figure  2.  Integration  of  Viper  Sensors  with  NRL  Sparrow  UAV 


Figure  3.  Downlinked  Video  from  Sparrow  UAV  Showing  Detected  Muzzle  Flash 


